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ABSTRACT 

that is the nature of value and how can one identify 
the value claiis made in evaluation documents? what is the proper 
philosophic basis for the 'treatment of values in evaluation? These 
and related concerns are addressed in this report. It is suggested 
that evaluation documents have a structure of claims or a sonceptual 
design, and that, by leans of philosophic analysis the philosophical 
assumptions say be revealed and the philosophical grounds for 
evaluation understood. Part 1 provides a systematic aethod for 
analysis of claim structures. The method is QOEBAi Value, a OEM AC is 
an acronym of six elements related to evaluation documents: question: 
unquestioned assumptions: event/objecv.: method; answer /claims: 
concept/conceptual -structures. The method and the associated 
technique of concept mapping are shown to be valuable in the 
construction of a document's philosophy and as aids to reveal the 
source and meaning of criteria of excellence. A Headstart program 
evaluation document is analyzed to illurtrate the method. Philosophic 
questions growing from concern with the key concept of valae are 
probed in Part II. Specific issues dealt with are: utilitarianism and 
evaluation: educational value: the distinction between "value" and 
"having values": and the virtue/function claim. (Author/AEP) 
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PREFACE 



The Research on Evaluation Program is a Northwest Regional 
Educational Laboratory project of research , development, testing, 
and training designed to create new evaluation methodologies cot 
use in education. This document is one of a series of papers and 
reports produced by program staff, visiting scholars, adjunct 
scholars, and project collaborators—all members of a cooperative 
network of colleagues working on the development of new 
- methodologies. 

What is the nature of value and' how can one identify the value 
claims made in evaluation documents? What is the proper 
philosophic basis for the treatment of values in evaluation? 
These and related concerns are addressed in this report by D Bob 
Gowin and Thomas Green. These two educational philosophers 
provide a procedure for analyzing the value claims made by 
evaluators and address such questions as "What is educational 
value?" and "What is the distinction between valuing and having 
values?" 



Nick L. Smith, Editor 
Paper and Report Series 
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THE EVALUATION DOCUMENT: PHIU)SOPHIC STRUCTURE 

Part I: 

t constructing the Philosophy in an Evaluation Document 

introduction 

Evaluation documents have a structure, in. fact, they have 
many different structures. As products of human effort, they tike 
different forms to serve different purposes. Some documents are 
used to help^clients, some are us^d to obtain funding, others are 
distributed to ^ network of evaluation specialists. Take a 
selection of any number of evaluation documents and you will find 
they are put togetner inj variety of ways, they have different 
structures. By structure in this context we mean only 'the parts 
of the document in relation to other parts which together consti- 
tute the whole. Structure is simply parts-and-their-relations. 
In particular, we believe that evaluation documents ha*e a 
philosophic structure, a structure of a special Kind. That 
structure is a structure of claims . Through philosophic analysis 
we make this structure of claims etfjplicit and we riveal philo- 
sophical assumptions. When we come to understand the philosophic 
structure of evaluation documents we come to understand the 
philosophical grounds for evaluation. 

We are no* trying to write a philosophy of evaluation as one 
might write a philosophy of science or education or art. We are 
writing this monograph to achieve two basic aims. First, we will 
provide the reader with a systematic method to use in analyzing 
the structure of claimB found in evaluation documents. Prom the 
use of thin method of analysis one may expect to be able to 
construct the philosophy found in the evaluation document. This 



construction should be immediately useful in two ways. It should 
help in the analysis of completed studies and in the creation of 
new studies. Further, it should help to show the source and 
meaning of the criteria of excellence used in judging existing 
studies and in guiding the developmept of new studies. The method 
and its use on a clear case of evaluation (Head Start) constitute 
Part I of this work. 

In Part II, "Further Questions About Value", we go beyond the 
results of an analysis of a document to probe more deeply f^r 
philosophic questions growing out of concern with the key concept 
for evaluation, value . We discuss a view of classical utilitar- 
ianism as a basis for evaluation practice. Next, we. take up the 
difficult issue of educational value. We believe that educational 
evaluation should relate to the value of education. Next, we 
extend the discussion to make & very important distinction between 
value and "having values". We end with another value question, 
found in the tradition of Aristotle's views on excellence. 

The Method of Analysis; QUEMAC Value 

Evaluation is defined as the assessment of worth, the 
determination of merit, the appraisal of value. Value is the key 
concept in evaluation. Yet it is surprising how infrequently the 
concept of value Is discussed in .evaluation studies, QUEMAC Value 
concerns value. It is a method we use to analyze and critique 
the structure of claims found in evaluation documents, QUEMAC is 
an acronym of six elements related to questions we can ask of any 
evaluation study. These questions, when answered, give us a sense 
of the whole structure of claims in the evaluatio" study. This 1 
structure is the pattern of concepts showing what has to be 
thought about to make sense of the evaluation. Perhaps we could 
call it the "conceptual design" in order to distinguish it from 
the usual concern of methodology for design devoted to establish- 
ing matters of fact (data, variables, measurements, etc.). 



The letters of QUEMAC stand for the following: 



Q 


stands 


for 


Questions 


U 


stands 


for , 


Unattended Assumptions 


£ 


stands 


for 


Events of Interest 


M 


stands 


for 


Method of Work 


A 


stands 


for 


Answers to Questions 


C 


stands 


for 


Concepts 



By centering on value questions, we also find ways of making 
sense of evaluation work. We have found it useful to look for 
the following five forms of value questions* and value claims. * 

Intrinsic value claim 1)' Is this X any goad? 

Instrumental value claim 2) What is it good for? 

Comparative Claim 3) Is it better than something 

else? 

.7 

Idealization claim ( 4) Can we make it better? 

Decision claim „ 5) Ought we to choose it? 

Almost all of the value claims ^valuators ar* concerned with take 
,one of these five forms. Notice that three main ideas have 
already been presented. ^jFirst, we are going to present a method 
of analysis.. That method consists of asking and answering the 
questions of QUEMAC. Secondly, we are going to show that the 
method of analysis reveals the, structure of claims in the evalua- 
tion document. Thirdly, notice that this method of working will 
/get us to the concept of value . 

Starting Points and Assumptions 

We begin an analysis by supposing that anyone interested in 
evaluation can find a clear case of an evaluation document. Take 
some evaluation study you are familiar with and, put it on the 
table in front of you. The clear case we use is an evaluation of 
Head Start. The first move in the analysis is to decide to locate 
(or infer) the important question of the study anci the main answer 



given in the study. The relation betweeh questions asked and . 
answers given is a key useful in unlocking the structure of any 
evaluation study. (See a later part of -this monograph for the 
worked out example of the Head .Start evaluation.) 
Notice already what we hive assumed: 

1. We assume that you can recognize a clear case of 
evaluation. This presupposes criteria you use to separate a 
clear from a mixed Case or phony case. Knowing what counts as an 
evaluation probably can commit you to a point of view about 
evaluation. And any "point of view presupposes a context in which 
that viewpoint is meaningful. v ' 

2. We assume an evaluation study generates a document (a 
deliverable) that ,you can put your hand o* and put it on the 
table, i.e., that a document is ajnaterial thing that takes' up 
space and doesn't disappear rapidly in time. Evaluations that 
result in conversations (e.g. r between a principal >and & teacher 
or a teacher and a parent or pupil) do not leave a material record 
that can be put on a tabled 

3. We assume in some evaluations no clear question is 
explicit. Answers are given as facta and data are presented to 
.the reader to find the questions. The reader then has to realize 
that answers can be converted into questions. L€*t us now go to 
an overview of QUEMAC. 

Overview of QUEMAC V * 

Q Quest ion t What is- the significant, telling question (TQ) of 
the study? Distinguish Celling .from technical 
questions; telling questions "£ell on" the context 
assumed or made explicit by the study. (See Note A 
for more discussion of TQ's) 
p unquestioned Assumptions t Identify assumptions taken for 

granted by the workers of the study. These assump- 
tions? ,too, will be related to the context in which 
the study was conducted. 



A 



E EverU/ObjfeCt : Describe brie fly the main event: is it a 

- program? personnel? product (e.g*, an object such 
as a textbook)? What is\he evaluation study about , 
what areUts characteristic phenomena of interest? 
Provide' a sucoinct statement of what was inquired 
* into. A process ist an event. 
Method : A method is a .procedural commitment. * We use methods 
to get answers, to our questions. We want our a 
methods to hav«, certain features fe.g., reliability) 
and studies in research design and methodology 
, generally are studies^of the most desirab.\e features 
of methods or£ work. Recent studies in alternative 
methods 'of evaluation are producing a much greater 
variety of approaches than were thought about here- 
tofore. A question of method is .a "How" question^ 
those skilled in methods and methodology are said 
to* have "know how*. 

Philosophic studies also use a set of methods to 
answet philosophic questions. These methods of ^ 
analysis are very different from standard research 
methods. We characterize the difference briefly in \ 
this way: philosophic methods deal with concepts 
i where research deals with facts. Philosophy ^ 

concerns concept analysis, not flata ar lysis. « f 
, ^ Empiricaj. method describes the ways, records wer$ 

m£de. Judge to see -if the records were made of the ^ 
key events or happenings of the program. Record- 
making and record- keeping are primary concerns of ^ *f 
systematic method. 

Data are transformed facts, that is, judgments 
based on principles bf measurement, statist^ps, 
design, etc. Data when transformed are taken as 
evidence to back knowledge claims. (The primary 
knowledge claim is af causal statement. Causal links 
are extremely rare and valuable. They., serve -as the 
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standard against which we judge pseudo-causal 
- statements (e.g., statements of correlation)]. The 
transformation of facts into data and data into 
knowledge claims is part of the work of method. 

s 

A Answer /Claims : What answer was given to the significant 

questions? What key facts did the method produce? 
j Which questions did the v investigators/evaluators 
knovf they failed to answer? Which answers did they 
* give to questions they failed to ask in the 

beginning? 

What critical limitations were the authors aware 
of? Unaware of? What claims beyond answers to the 
original questions do the authors make? 
C Concept/Conceptual Structures: List the key concepts (no 

more than 10); describe the apparent ordering among 
these concepts. Which two or chree, key concepts 
are present in the significant questions? Try to 
identify the pattern of reasoning, the main 
arguments, the logic pf the study. 

Some concepts subsume others. Arrange the 
concept:* in an order with the most powerful ideas 
at .the top of a sheet of paper and the subordinate 
ideas toward the middle of the page. The opera- 
tional concepts, those closest to the events of 
interest, will appear toward the bottom of the page. 
We call this ordering of key concepts a concept 
cluster or concept map. ; 

Which key concepts are defined 1 ? Is the 
definition intelligible and useful in the context 
— in whiqft it appears (i.e., what definition is 
given?)? 

TO find key concepts, look fort operational 
definitions; the meaning given to independent/ 
dependent variables; the theory or model or 
schemata, etc. ;, £he *ey references. 

6 
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In brief: 


Q s 


nudt* 19 tile v^ucBUivJiif 


U: 


wnat is asjsumea, unyvcov.iuncuf 


Eju 


What is the quebrton aoout? (Tne event, oojectj 


M: 


How is the question answered? (methodology) 


kC 


What (are) the major answer (s)? 


C: 


What conceptual structure is used? 



Answering these six main questions Will produce a view of the 
structure of the evaluation s^tudy. Because QUEMAC appraisal is 
alsjo a critique of an evaluation, we call it a meta-evaluai. Ion. 
It is "meta-evaluation" because it^is a kind of knowledge about 
evaluation that jie generate through the analys: The words 
"QUEMAC Value" help to focus the critique on the value questions 
of the evaluation. 

< 

A Clear Case Analyzed; Head Start 



K<- lan, Eleanor S., "Head Start Experience and the Development 
of Skills and Abilities in Kindergarten Children" 
originally appeared in Graduate Research in Education , 
April, 1966, pp. 4-28; see also Millman, J., and D. Bob 
Gowin, Appraising Educational Research , Prentice-Hall, 
1974, Chapter 4, pp. 23-49. 

This study was published in 1966. The author wrote the 
following summary^of the studv. After vgp read the summary, then 
read^the QUEMAC appraisal applied to the study. This appraisal 
of necessity leaves out much detail. For a more complete 
critique, see Millman-Gowi , Appraising Educationa l Research, 
chapter four • 

Summary t Head Start Evaluation 

The purpose of the study was to assess whether the 
children who participated in Project Head Start were 
better prepared for kindergarten than those who did not 
participate in' regard to verbal communication, visual 
discrimination and naming,, and motor coordination skills. 
It was hypothesized that kindergarten children who 
'participated in Project Head Start would be superior in 

i 
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verbal communication abilities as measured by verbal 
fluency, verbal usage, enunciation, ability to structure 
sentences, and ability to tell a story; that they would 
be superior in the visual discrimination and naming of 
colors and shapes; and that they would be superior in 
motor coordinator skill, as measured by 'drawing figures, 
coloring, cutting, and buttoning their clothing. 

The subjects were seventy kindergarten children between 
the ages of four and five who wer? identified on the basis 
of whether or not they had participated in the Head Start 
program during the summer of 1965. Thirty-five children 
who had participated in this project were paired with 
thirty-five children who had not participated in the 
project using sex, age, ethnic background, language spoken 
in the home, age of siblings, and preschool experience a* 
the criteria. 

The subjects were then compared with respect to their 
verbal communication abilities. They weie asked to tell 
everything they saw in a sample picture and to tell a 
"Story about that picture. They were rated according to 
the ability to structure sentences, story-telling ability, 
verbal fluency, verbal usage, and enunciation. 

The subjects in each group were "also compared in their 
visual discrimination abilities. Four colors and three 
shapes were presented to each child who was then asked to 
name each color and shape. The children's productions 
provided data to evaluate their cutting, coloring, and 
drawing skills. The children's ability to button their 
own clothing was also observed. 

The data were analyzed in terms of comparison between the 
Head Start and non-Head Start groups, utilizing chi- 
square median tests of significance. The Head Start group 
did significantly better than the non-Head Start group in 
all hypothesized abilities and skills except for enuncia- 
tion ability and cutting skills which did not reach 
statistical significance. 

The present findings support the current view that 
culturally deprived children benefit from preschool 
enrichment programs. It was suggested that future 
research should further examine the values of preschool 
compensatory programs and establish an appropriate 
curriculum. Longitudinal studies are needed in order to 
ascertain the long-term benefits of such a program. 
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An Example: QUEMAC Value Appraisal of Head Start 
Q Question (s) : 

(1) "Are children who participated in Project Head Start 
better prepared for kindergarten than those who did not 
participate?" 

(2) ''Can racism in America be combatted through the means of 
schooling?" 

U Unattended Assumptions : 

(1) "The earlier in the child's development, the better for 
schooling" ("effective enrichment"). 

(2) "Nurture, more than Nature, controls child development." 
(i.e., Jensen is wrong.) 

E Event ; , 

(1) A summer long Head Start program (e.g., "show and tell", 
"using letters of the alphabet",, "classifying objects", 
"drawing the body shape of each child on brown paper", 
"distinguishing colors and shapes", "buttoning") 

(2) Events made'to happen by the evaluators (e.g., "teUing 
a story", "structuring sentences", "naming colors").. 

M Methods : 

Records of events : Various testing devices — (e.g., "Describe 
this picture test" - two firetrucks going to a fire, two 
policemen guiding traffic, people watching, a dog, etc., - 
and each child told a story until each child insisted there 
was nothing more to say) 4 

Data transformation : e.g., children rated according to 
complexity of statement about the picture: A numerical value 
of zero assigned if nothing was said, of one if picture was 
described, of two if picture was described in considerable 
detail, of three if unorganized story was told, of four if 
organized creative story was told. 
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Table 1 



— «t 

V«' al Abilityt 


Number 


of Children Receiving 


Telling a Story 


Each Rating 


Story-telling Rating 


Head Start 


Non-Head Start (35) 


4 


10 


4 


3 


11 


7 


2 


6 


10 


1 


7 


11 


0 


1 


3 




Median - 3.3 


Median » 2.3 



• Chi Square » 5.74 
p < .025 



A Answer/c^ims : 

(1) (e.g.) Head Start children were superior— in story- 
telling ability, especially in telling creative, 
organized stories . ♦ . to -non-participants. 

(2) "Culturally deprived children benefit from preschool 
enrichment programs . . . ." 

(3) "Longitudinal studies are needed to ascertain 
long-term benefits"." 

C Concepts/Conceptual Structures : 

The key concept that guided the "treatment*" given in Head 
Start programs is language. The relation between language 
and social class was 'described in an influential series of 
publicatibns represented by Basil Bernstein 1 * "Language and 
Social Class", British Journal of Social Class , 1960, 271-276. 
Other concepts were also significant, especially the normative 
claim for equality of educational opportunity, cited in the 
study as "effective enrichment". But, as you can tell from 
the way the main hypothesis was stated, what is more central 
to the thinking a^ this time was language i verbal fluency, 
usage, enunciation, sentence structuring, telling a story. 
So we may construct a concept cluster, or concept map* 

This example of a concept map shows the cluster of key 
concepts ranked in order from general to specific (thbse 
closer to actual regularities in events which happened in the 
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study) . The concept "effective enrichment" is given 
operational meaning on the assumption that "more is better" 
when„it comes to language use. 

As the key concepts move down toward actual events , there is 
a loss of abstraction, a gain of precision, and an increased 
likelihood the events of interest will be trivialized by the 
selection of indicators, such as operational definitions. 
This shift in the level of discourse is a basic point to note 
in making a critique. 

Thus, QUEMAC appraisal has a logic which connects the 
separate items of appraisal. It does not matter which question 
is answered first just so long as all questions in QUEMAC are 
answered and the critical abstract constructed out of the answer. 

In brief, in this Head Start evaluation report, the children 
who were in the summer program did better on a number of tests, 
primarily those of verbal communication, than did children who 
did not participate. Little description of the actual curriculum 
and instruction was provided. Alternative possible explanations 
of che findings were not given, nor was there adequate 
description of such concepts as "effective enrichment", "lower 
social classes", "racism", "equality of educational 
opportunity". The cost of the program and comparison to critical 
competitors were not described. The main conceptualization was 
derived from studies in language and social class. 

Using QUEMAC Value 

General . In general we are asking value questions of 
evaluation eports. We want to 4now if the evaluation is "any 
good". So we can ask: "Is it good? What good is it?" The 
analysis of the evaluation report takes us through its several 
parts and leads us toward a fir^l synthesis, an appraisal of its 
good ne s s /badnes 3 • 

Most evaluation stodies will not be organized so as to reveal 
their telling questions, or typically, their value questions. We 
believe that evaluators should ask and try to answer all five of 
the value questions shown below. Meta-evaluators should do the 



CONCEPT MAP 



General 



effective 
enrichment 



Language 



(as repreeented in speech patterns of) 



(through 
Heed Start) 





lower social 
classes 









has 



grematically 
simpis 



Suggested treatment in kinder- 
gsrtsn through language exer- 
ciass 



tslf 
$ton 



by 



incomplete 



syntax 



limited, rigid 
uss of sdj ac- 
tives, adverbs 



and 



verbal convruni ca- 
tion skills 



name co 


tors & shspss 


etc. 





juperst tonal I v define 
ths main variabias as 



such as 



strucvving 



fluency 



enunciation 



number of 
words used 



structuring 
sentences 



rated by 
taachers 



Specific 



count tha 
number of 
adjectives, 
adverbs 



storytelling 



[Tsted by 
taachers 



rated by 
taachers 



is 

12 



same. These questions help us to see what is NOT in the evalua- 
tion report. A critique of a study will show us what is missing. 



Five Fundamental Forms of Value Questions and Value Claims* 



Our experience with QUEMAC Value appraisals has led us to 
formulate five forms of value claims. Each of these forms 
may be phrased as a question. And the obviously appropriate 
place in QUEMAC is the Question. Put differently, evaluation 
studies pose their telling questions as value, questions. 
What forms do these value questions take? 

Example: Head Start 



Forms of Value Questions 

1. Instrumental Value Claim . 
"Is X good for Y?" 



2. 



3. 



4. 



Intrinsic Value Claim 
"Is X good?" 

"What is the good of X?' 



Comparative Value Claims 
"Is Y better than X?" 
"Is X better than Y?" 

The Decision Claim 

"Is X right?" 

"Ought we to choose X?" 



5. Idealization Claim 

"Is the good of X some- 
thing that can be made 
better? 



Is Head Start good for something 
else? 

Is it good for future academic 
achievement? 

Is it good for community 
relations? 

Is Head Start good? 
Good in itself? i.e.. 
Head Start is good if educative. 
How is Head Start educative? 
What is it about Head Start that 
is educative? (i.e., what are 
the treatments; how do they 
work?) 

Is Head Start better than no 
treatment? 

Do Head Start pupils do better 
than non-Head Start pupils? 

Ought we to choose Head Stajt? 

Is Head Start right? 

Is compensatory education right? 

How could Head Start become 
better? 

What is the wood in Head Start we 
can idealize to make it better 
than it is? 



*This list of value questions is not exhaustive, 
for further discussion. 



See Part II 
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How to Find These Value Questions {See Note B for more ciiscusaion) 

1* Instrumental value questions . Since it appears the 
easiest value question to analyze , meta-evaluators should begin 
here. Thus, for example , the question "What is Head Start good 
for?" leads immediately to instrumental value claims, e.g., "It 
is good for Keeping children in school." 

2.. Intrinsic value questions . One way of finding intrinsic 
value claims is just to list all ttys instrumental value claims 
first, and see what is left over. Many instrumental value claims 
are directly linked to the intrinsic goods t:iey serve. 

A second way to find intrinsic value is to analyze the full 
and complete description of the object (event) being studied. 
Not all descriptions are descriptions of the value aspects of the 
object (event) , but complete descriptions should include them. 
The intrinsic value of Head Start is its direct educative value, 
i.e., that some children were taught some things that educated 
them. 

Mixed bag . JJsually we find. most objects of evaluation to be 
both intrinsically and instrumentally good. 

If one is making an evaluation study, one question should be 
asked: "What is the good of X?" If it- is a program, or a 
product, or a position, then those concerned with its evaluation 
will typically have some idea of its worth. Becoming explicit 
about what the values are in the events-objects is a direct help 
in getting at the value behavior of the relevant -participants/ 
clients/audience. In met«*evaluation, one should focus on whether 
the primary evaluators asked the question about the intrinsic 
value of the program, product, or position. 

3. Comparative value questions . These questions are easy to 
find because typically they are explicitly asked in the evaluation 
study. Comparing one thing to another is a basic activity in 
evaluation. In thia case, groups of children were compared. 

4. Decision value questions . These, too, are easy to spot* 
especially when evaluation is conceived to be a service to a 
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client who must make decisions. No decision-question was asked 
in this Head Start evaluation. * 

5. Idealization valuta questions . Sometimes the gap between 
the real and the ideal is made explicit. Filling this gap is 
thought of as a way tc meet basic needs (i.e., the gap between 
ideal-real is a definition of "need"). No idealization question 
was asked in this Head Start evaluation. 

QUEMAC Value Critique Summary 

The QUEMAC appraisal, when summarized, should provide a 
succinct structure of framework of the evaluation study. 



Figure 1. The Vee 



Conceptual Side 



Methodological -factual Side 



Conceptual structure 



Telling Questions 





Major answer - 
/ / Method 



Value Claims 



Knowledge 
Claims 



concepts 



Recofds-of events (facts) 



event 
object 
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The Vee is a heuristic device we use to bring' together the 
various parts which together constitute the structure of the 
study, we talk about "laying the Vee" on documents produced by 
studies. This means that we have a template, a guide, a form, £ 
device that gives us knowledge about knowledge: meta-knowledge, 
meta-evaluation. W* also manage to preserve complexity while 
simplifying the way we represent the analysis of claims. 
Criticism shows us the structure of claims. 

The simplest structure is the question-answer structure. What 
was the question? The answer? Since (tfestions and answers are 
something human* produce through systematic inquiry, the method 
of approach to them is important. And since the questions and 
answers are about something, we need to specify the event/object 
or phenomena of interest .that the study is about. 

And since we think with a small number of key concepts , 
abstracting the study so as to present the concept cluster used 
is a convenient and important way into the thinking of the 
evaluators. Identifying unquestioned assumptions is important 
since any study must make assumptions, even though those 
assumptions might be questions in prior or subsequent studies. 

Note A 

Telling Questions. Some Problems and Hints 

a. Only a- small number of telling questions should be 
expected (1-4). 

b. They may be ranked in order, but they need not be. 

c. Some telling questions are not found in evaluation 
studies; the only questions asked are technical questions. 

d. Some telling questions ar found, but not answered in 
evaluation studies. Sometimes (often unknowingly) evaluators use 
a smokescreen technique! a relevant telling question is asked to 
vindicate the study from charges of lack of significance, then 
this real issue is pushed aside and another more accessible 
question is answered. 
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e. A meta-evaluation can appraise the congruence between the 
telling questions asked and those which should have been Ssked. 

f. If you can't find the telling question, then look at the 
answer actually given. Not only does the answer help formulate 
the question exactly and clearly (on the principle you know what/ 
the problem is when you find the solution) , but sometimes 
evaluation studies start from an already defined conclusion. 

g. If all else fails,- a telling question can be built up 
within a study by using the journalists 1 technique* who, when, 
where, why, what, how. An example that signals many of these 
questions is "Can we improve nutritional practice in lov-income 
"families through an educational program for homemakers conducted 
by nutritional aides in the home?" These telling questions are 
constructed from all the concerns, assumptions, and information 
available within the study, and as such, it is an end product of 
the meta-evaluation analysis, not a starting point discernible at 
the beginning. Once you compose it, however, you should begin to 
believe it is the correct one, and that it pulls together in a 
succinct way the major concern of the study. 

h. Telling Questions and Context. Going outside the study 
to the context which gives the study significance is the most 
difficult and perhaps more important place to search for telling 

' questions. Knowledge of the wider context requires knowledge of ' 
■history, of social problems and their significance, arid of philo- 
sophical theories. Head Start needs to be placed historically* in 
the context of Lyndon Johnson's Great Society; the social problem 
of racis.it, and the philosophical view that government and educe- . 
tion can worU together to solve social problems. In general, 
ethics and social philosophy have been under-utilized as a way to 
make sense of evaluation theory, policy and practice. 
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Note B 



A Ubiquitous Vglue Question t "Is X good enough?" 

This question, "is X good enough?", thought to be typically 
American in formulation because of its pragmatic and expedient 
flavor, actually admits of interpretation in all the five forms 
of value questions we h*ve presented* The question has many 
meanings which must be distinguished before we try to specify an 
answer. This question can lead us to all the other value 
questions, and is, thus, a good starting point. 

1. Intrinsic value claim , "is X good enough?" translated 
into meaning "Is X good at air?" "is there any intrinsic value 
in X?" .Ms Head Start educative?" "How is it educative?" 

2. Instrumental value claim , "is X good enough, considering 
the ends we have, to serve as means?" "is Head Start good enough 
to keep pupils in school?" 

3. Comparative value claim . "Is x good enough, compared to 
Y?" "is Head Start good enough, compared to pupils who do not 
get it?" 

4. Decision value claim , "is X good enough, given contextual 
constraints, that we ought to choose it?" "TCs Head Start good 
enough, given racism and unequal educational opportunity, that we 
ought to choose it?". 

5. idealization value claim - "is x good enough, or can we 
make it better ?£. "is Head Start good enough, or can we improve 
on it Weally?" 

A Special Technique! Concept Mapping 

Telling questions contain at least two concepts. To identify 
the telling question, then, puts you on the way toward identifying 
the conceptual structure. By conceptual structure I mean simply 
the cluster of concepts and the way, in which they are related. A 
concept is a sign or symbol that refers to regularities in events 
and in records-of-events (the facts) . 
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Concepts, conceptual- structures, concept clusters come in a 
variety of forms. For example /^e conceptual structure of music 
for the piano has a. definite but loose form (Cf. A. Dyckraan) . 
Thi* form has been influenced by many things: the hi&torical 
traditions of music, changes in the concepts (of a scale: A. 
Schoenberg 12-tone scale, for example), the inventive grammar of 
creativity in music (Cf. L. Bernstein's lectures at Harvard), and 
the foul? major traditions (linear, atomistic , holistic, analytic) 
for teaching piano music (piano pedagogy) . 

The existence of different and definite conceptual structures 
in a field of study provides us wich the evidence we need to back 
the assertion that multiple structures ^exist in a.U intellectual ' 
fields: a discipline is one word but many things. So, when we 
speak of t^ conceptual structure in our analysisit is wise to 
remember that more than one such structure exists in any field. 
An historical review of the telling questions and conceptual 
structures in any fic^ld is a find way to see these structures. 

Concept Mapping 

Concept mapping is a technique of analysis. We use i% to show 
on a piece of paper just what a conceptual structure might look 
like. Once you have located the telling quest iohs, or h paragraph 
that appears to contain the key concepts, then simply rank these 
terms. Put at the too of a piece of paper the most important 
-terms. Rank just below th$m the next set of related but less 
important terms. Finally, toward the bottom of the pa^e place 
the terms that come closest to events. Draw lines of relation 
between these terms. When you £«Ve done that you have drawn a 
concept map. See Figures 2, 3, and 4. 

In Figure 2 you will see the first page of a classic^ research 
paper by Gregor Mendel, translated into English. It is difficult 
for non-specialists to read. In Figure 3 you will see a list of 
key concepts as they appear in the original. Such a list in 
itself does not make much sense, in Figure 4 you will see a 
concept map. This map organizes the concepts *n*the original so 
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that the most important, concept is placed at the top of the page. 
As you move down the page you fi ee how the various concepts are 
related to each other. At the bottom of the page, we come to 
specific events or objects. The making of a concept map is one 
way to see the structure of conceptual claims in a document. 

A concept map is a very handy way to represent ideas.. When a 
significant piece of material' ha. been converted into a concept 
map, then students can locate themsleves on the map. They can 
spot the terms they understand and see how these are related to 
new ideas they do not yet understand, students who have learned 
to use this technique can begin to draw their own maps. A. concept 
map is like a rubber sheet, when we pull at one place to highlight 
one set of ideas we find that other ideas shift. No one set of 
concepts is fixed for all time in a definite hierarchy. The 
Pattern of ideas has an elastic quality. 

Concept Definition 

. One of the most important things we do with key concepts in a 
field of study is to give them a definition. ' Defining what we 
mean by our key terms is a significant pedagogical move in making 
ideas usable by others. It is better, Charles Pierce wrote, to 
have a few clear ideas than a heedful of fuzsy ones. Finding the 
definitions of the key concepts in a discipline (and knowing what 
kind of definitions they are, is a central move in curriculum 
analysis. 

Over 20 kinds .of definition have been identified by 
Philosophers (Cf. Robinson, Definition,, j an concerned here 
only with concept definition. Concept definition is similar to 
operational definition. The similarity is found in the relation 
between words and events. Basically to define a concept is to 
show the way the key term points to the regularities in the 
Phenomena of interest, the selected events. Dictionary 
definitions typically are word-word definitions. That is, they 
define one word in terms of other words. At some more abstract 
levels of conceptual structures this word-word definition works 
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< Figure 2 

EXPERIMENTS IN PLANT-HYBRIDIZATION 
Gregoiv Mendel f 



Introductory Remarke* 

Experience of artificial fertilieation, auch aa ia affected with ornafl&rftal 
planta W ordsr to obtain naw variationa in colour, haa lad to the experiment a 
which will bare be diecueeed. The etr iking regularity with which the ease 
hybrid foria alwaya reappeared whenever fart ilieat ion took plsce between the 
saa* species indeced further experimente to be undertaken, ther object of which 
waa to follow up the developments of the hybride in their progeny* 

To this object numerous careful observare, euch aa Kolreuter, Gartner, 
Herbert, Lecdq,l Wichura and other t, have devoted a part of thsir livee with 
inexb*uetible'pereeverance. Gartner especially, In hie work "Die Baetarder**** 
gung ia Pflansenreiche" (Ttoe Production of HybrMe in the VegeUhle Kingdomy, 
haa recorded very valuable observations f and quite recAtly Wichura published 
the reeulte of some profound investigations into the hybride of the Willow. 
Ttiat, no far, no generally applicable law governing the formation and develop- 
ment of hybride has been successfully formulated can hardly be wondered at by 
anyone who ie acquainted with the extent of the task* and can appreciate the 
difficulties with which experiments of this class have to contend. A final 
decision can only be arrived at when we shall have before ue the reeulte of 
detailed experiments made on planta* belonging to the moat diveree orders. 

Thoee who survey the work in thie depart~-nt will* arrive at the conviction 
that among all the ,numeroue experiments made not one haa been carried out to 
auch an exttnt ariS' in auch a way aa to make it poeeible Co determine, the number 
of different forma under which the offspring of hybrids appear, or to arrange 
^heie forma witii csrtsinty according to their separate generationa, or 
definitely to ascertain their statistical relations. 2 

'It requires Indeed some courage to undertake a labour of such far-reaching 
extent; thie appears, however, to be the only right way by which we can finally 
reach the solution of a quiet lev the importance of which cannot be overesti- 
mated in connection with the hietory of the evolution of organic forms. 

The paper now preaented records the reeulte of such a detailed 
experiment. Thie experiment waa practically confined to a email plant group, 
and is^ow, aftwr eight years* pursuit, concluded in all eseentiale. Whether 
the plan upon which the eeperate axperimente were conducted and carried out 
waa the bees euited to attain «tt deeired end ie left to the friendly decision 
of the reader. / 



Selection of the Experimental Plaftta * 

Tn- value and utility of any experiment are determined cy the fitneee of 
the material to/ the purpoee for which it ie ueed, and thua i,i the caae before 
ue it cannot Joe immaterial what planta era aubjected to experiment and in what 
manner euch experimente are conducted. 

The eelection of the plant group which ahall serve for experimente of thie 
kind muet be made with all ... • 



\ 

iThia tranalation waa made by the Royal Horticultural Society of London, 
and ia reprinted, ->y permieeion of the Council of the Society, with footnotee 
added and minor changae auggeeted by Profeaeor W. Bateeon, enclosed within i ). 
The original paper waa publiahed in the Verb, naturf. Var. in Btunn, 
Abhandlungen , iv. 1865* which appeared in 18*6** 

2| It i, to the clear concept/on of these three primary neces«itiea that 
the whole succeee of Mendel'e work U due. So far aa I know thia conception 
waa abaolutely new in-, hie day.} 
«5 . ' 
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just fin*.* But at some point, every conceptual structure needs 
to be brought into relation to the real events, or phenomena, of 
interest. Getting an "event sense" of an abstract concept is a 
good step to take toward understanding it (ostensive-pointing- 
d "initions) • 

Operational Definitions 

Operational definitions appeared in modern physics and 
philosophy in the first quarter of the 20th century. Because 
there were difficulties with certain very abstract concepts 
(light, length), some physicists (Percy Bridgeman was a leader 
here) recommended that the key concepts could be defined by 
discussing the way they were used in the activities of measuring. 
So, length became operationalized by discussing whether one used 
a yardstick or the speed of light as the measuring instrument. 
In psychology, intelligence was operationally defined by the 
operations of giving and scoring intelligence tests. The record 
of the event of test-giving, test-taking, and test-scoting was 
the observation. Intelligence was then defined in conjunction 
with the operations used to measure it (Cf. A. Rapoport, 
Operational TPhilosophy ) . 

Concept definitions are similar to operational definitions, 
as I have said. The similarity consists in the common focus on 
events and their records. But it is not necessary to reduce all 
concepts to operations just so long as some thread of connection 
can be made between the most abstract of concepts and the regu- 
larities of the events of interest. Our technique for concept 
mapping shows us very clearly just how these relations between 
the abstract and concrete can be drawn in different studies. 

Concepts refer to patterns, regularities in events. When our 
concepts are well defined it seems that the events are maximally 
invariant. The platinum meter stick on exhibit in Paris is taken 
to be the definition and the standard of precision for all meter 
sticks; it varies very little. In ordinary experience most of us 
do no*: have any trouble defining a car as a car because, even 
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though there exists a large variety (the variance is great) , the 
regular features of a car (four wheels, a motor, gears, etc.) are 
quite common and familiar to us. Likewise, in the conceptual 
structures of a discipline, there will be some concepts with 
precise definitions and others with much l--ss precision. 
Precision and clarity of meaning are both important. 

Concepts and Constructs 

Conceptual structures in working sciences carry a share of 
constructs in addition to concepts. Constructs are simply those 
ideas that help to hold togethger the key concepts. These 
constructs are like bridges across one conceptal ground to 
another. They are constructions between concepts. In genetics, 
for example, Mendel used the construct of "factors" which later 
gave way to "gene". These ideas did not have an operational 
meaning. As gene has now changed in some biochemical studies of 
genetics to "polypeptide" (one gene- one polypeptide is the 
central dogma today) , the status of ttje word has shifted from 
that of a fertile construct to a definite concept. 

Concepts and Logical Operators 

All conceptual structures also carry logical operators. These 
are terms like "either", "or", "not both", "and" (meaning conjunc- 
tion) , "If • . . then", and the like. These terms help in framing 
conceptual arguments. Well-reasoned arguments are a most signifi- 
cant part of any field's conceptual structure. In some cases 
these arguments can almost be reduced to logical forms (e.g., "not 
both p and g"). The closer one moves to simple logical purity, 
however, the farther one moves away from substantive assertions* 
of claims about the world. It is important to recognize logical 
operators and logical arguments in conceptual structures, but it 
would be a mistake to substitute one for the other. 
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It is convenient to think of concepts as words. The word 
"time" and the word "tiempo" have the same meaning. "Time" is 
part of the Fngliah language and "tiempo" Spanish. The shape of 
the words are different, and the sounds are different. What, 
exactly, is it, then, that is the concept? 

The first move to make with this question is to suggest that 
the words are carriers of the concept. We call them sign- 
vehicles. Just as an automobile and a wagon are vehicles for 
passengers, words are vehicles for concepts. But this analogy, 
helpful as it is concerning the carrier role of signs, is 
misleading because it suggests that concepts are entities taking 
up space and time. This suggestion would lead us to try to locate 
concepts, perhaps as mental entities in the mind, or physical 
entities of some sort in the brain. 

But concepts are more like fluid events than fixed entities. 
When we think of words as carriers of concepts, then we can also 
think that the word-vehicles (including terms, sentences, symbols) 
serve the role of being concept facts. That is, words are records 
of concepts . A word is a record of a concept event. We use words 
as the facts of language. 

"taken . A word is a vehicle for a concept the way a token (a 
piece of money, e.g., a dime) is a carrier of value. We exchange 
token- in barter and other free market activities. Words are the 
medium of exchange in free communication; we exchange concepts in 
a conversation.* 

In concept exchange nothing is lost . One of the most 
significant points about exchanging ideas with another person is 



♦ Type and token . Linguists and philosophers make a 
distinction between a word type and a word token. For example, 
on this page of printed material the word "the" is printed more 
than one time. By counting all the times the word "the" appear*, 
you will have the number of tokens of that word. Each time the 
same word appears it gets counted. So far in this paragraph, 
there are seven tokens for the word "the". But the word "the" as 
a word type has appeared only once. 
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that nothing is lost, and much is gained, A slight amount of 
energy is used, perhaps, but we scarcely notice it. If I give 
you an idea, and you give me one* in return, I do not lose my idea 
and I do gain yours. Teaching, as the achievement of shared 
meanings, is economically very efficient. 

Review and Summary 

In Part I we have presented a method of analysis, called 
QUEMAC Value. Our experience in using this method has made it 
better but it could be improved. As you experience this mothod 
you may see a need to change and improve it. We would appreciate 
receiving suggestions if you care to share them with us. 

The concept of value is, we think, fundamental to the practice 
of evaluation. Like many basic concepts there exists a cloud of 
ambiguity surrounding it. In Part II of this work we try to 
clarify the meaning of the concept of value. We next turn to 
this task. 
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Part Hi 

Further Philosophic Questions About Value 



Utilitarianism and Evaluation 



Philosophic theories of ethics and social philosophy propose 
answers to questions about what is good and what is right. 
Evaluation practice also involves questions about what is good 
and what is right . The good and the right become significant for 
evaluators because evaluation studies are structured to help 
clients reach decisions. In making a decision we are trying to 
decide what we ought to choose, and why. Decision-making is 
supported philosophically by claims about the good and the right. 
What the right thing for an individual person to choose and do 
may not be the right thing for social order. The question, then, 
of sovereignty, or ultimate authority over actions, becomes 
important. Concepts of authority (power over) and social utility 
(what is useful for social problems common to the group, e.g., 
safety, survival) need to be related to each other. 

h fundamental question is how do we reach agreement about 
social policies and social actions. This question concerns ways 
we secure cooperation among people so common purposes can be 
shared. Such common purposes, for example, are safety, rule by 
law, health, education Securing cooperation requires us to 
search for a common ground among people. We need ways to reach 
agreement from each of us in sharing in the common purposes. Each 
of us has a stake in the whole, and the whole social order is 
governed so common purposes are achieved. When ? common ground 
cannot be found, we can often agree on a procedure for deciding 
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what to do without having to agree on what is good. One of the 
goods, then, is a good procedure. 

Questions about goodness-badness are value questions. Answers 
to these value questions we call value claims. Value claircs are 
like knowledge claims in many ways; value and knowledge claims 
are mixed together in evaluation studies. We can begin to sort 
t^hem ouc by an awareness of background ethical and social philo- 
sophy theories. We accept the view that ethical theories offer 
assistance for individuals in deciding what a person ought to 
choose and do, and that social philosophies propose different 
views for what a social order ought to be like and why. Individ- 
uals are persons-in-a-social-setting so the dividing line between 
the individual and the social is not hard and fast. 

A key issue for evaluation practice is the relation between 
the individual and the social. Can we satisfactorily evaluate a 
program (say. Head Start) by getting data on individuals and 
summing those data as an index to the social? Or must we assess 
\a program in terms of its social structure and loajLat the quality 
of the educative event as such? We know that schooling events 
have different qualities. We know that a specific act by a 
teacher in one school may be educative and the same act roiseduca- 
tive in another school social setting (Cf. Rugger, et al., 15,000 
Hours ) . The same act may be educative for some persons more than 
others. 

This key issue is also important in ethics and social 
philosophy, social philosophies take many forms, and what we next 
write is a simplification, but we think not a distortion. We 
sketch three major, social world viewst individualism (classical 
liberalism) , collectivism (socialism) , and totalitarianism 
(fascism). Classical liberal individualism takes the view that 
the (atomistic) individual is sovereign and that a benign social 
order develops as a by-product of individuals pursuing their own 
well-being. The social order is nothing but a. collection of 
individuals. Socialism takes the view that an organic unity 
between individuals (such as family, or team) is sovereign and a 
good social order develops as cooperation is pursued as 
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intrinsically good interactions among people. Fascism takes the 
view that the total state pursues the interest of an intrinsic 
total good and individuals derive personal worth from participa- 
tion in something greater than themselves. The main point in this 
paragraph is to note that these different views on the relations 
between individuals and the social order help us to see alterna- 
tive assumptions about what is to count as a primary good in the 
realm of the social order. A primary good is one which is 
fundamental to the development of other goods. 

Utilitarianism> a Philosophic Basis for Evaluation 

He think the philosophic view known as classical utili- 
tarianism influences evaluation practice. We will show how 
utilitarianism is a plausible interpretation of both purpose' and 
procedures of evaluation. In particular* the focus on evaluating 
consequences of social policies, a principle of utilitarianism, 

highlights the structure of evaluation documents, ana we believe 

# 

suah abstract views help to explain why evaluation practice is as 
it is today. 

The hope of the classical utilitarians Jeremy Bencham and John 
Stuart Mill concerned procedures for making value judgments 
objective. They wanted to be able to judge social policies and 
social actions in nineteenth century England. They thought such 
policies and actions could be objectively judged by evaluating 
the consequences of such actions and policies. And they were 
convinced that the effects of actions could be counted (measured). 
Quantification had appeal because they thought it was a sur~ way 
to find common ground for social agreement. Agreement betv ~<n 
persons, sometimes called intersubjective agreement, is desirable 
in democratic social orders. Evaluations of social actions which 
result in common agreement are also highly desirable* It is 
important to stress that quantification should serve the interest 
of reaching a common ground about what social actions are of 
value. Quantification as a basis for scientific generalizations 
is not the important point. Utilitarianism is an ethical theory 
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proposing answers to the normative question: "What makes an 
action or practice right? wrong?" 

A major assumption of classical utilitarianism is that each 
person is to count as one and no one's good can count for more 
than anyone else's (one man, one vote)., This principle of 
egalitarianism is taken to apply to all-, and is, therefore, 
universalistic. That is, the consequences of a social action that 
might affect anyone are to count as having the same social value. 
Each person is to count as one, and any social policy or action 
can be justified as it promotes the greatest good for the greatest 
aggregate number of persons. The greatos^ good is the best 
balance of good compared to bad consequences of social actions. 

How 38 utilitarianism propose to assess the value of states 
of affairs that are the consequences of action's or practices? 

How do we evaluate the consequences of social policies r 
actions, practices? And, how is it that evaluation practice 
checks out the differences in consequences? If Sesame Street or 
Head Start can be taken as instances Qf social actions, what 
should«evaluators do to assess their merits? What sort of 
consequences of these social actions are good? bad? 

We must realize that some states of affairs are intrinsically 
good. Some things are just good-in-themeelves. Simple pleasures r 
good health, the happiness that comes from intellectual grasp can 
be taken as positive instances. Absence from pain, disease, 
poverty, intolerance are also instances of goods. Freedom ' 
(liberty) is a primary good. It is the marginal balance of good ^ 
over bad that must be assessed in any giverv case. We may not 
always agree on all states of affairs that are intrinsically good, 
but most people will recognize in particular cases the existence 
of what can be taken as intrinsically good. If being in charge 
of one's own education is intrinsically good, then educating 
people is a right action on utilitarian grounds.^ Educational 
value accrues to individuals in a social context. 

in utilitarianism, any action or practice is right if it leads 
to intrinsically good states of affairs. The social action of 
Head Start is right because it leads to educationally powerful 




goods. In evaluating social actions, then, we look to the 
consequences of these actions as they conduce to good states of 
affairs. He might ask in our evaluation "What do Head Start 
practices bring about that is intrinsically good?" What different 
sorts of assessments are appropriate to establishing the Worth 
(goodness-badness) of Head Start, or any other social action? 
Since actions are right that lead to good states of affairs, 
therefore, we look to the consequences of actions to see if they 
are good or not. 

In classical utilitarianism two views have been presented 
about what is intrinsically goods pleasure (Bentham) , happiness 
(Mill), in Bentham total balance of pleasure over pain was what 
counted. In Mili the higher pleasures of intellectual excellence 
contributed more than sensual pleasures to happiness. Fot both 
Bentham and Mill the correct assessment of the good was a major 
concern. Can pleasures be counted? Bentham proposed a.hedonic 
calculus, a snethod for assessing pleasures. Mill thought the 
good, seen as happiness, is not the mere sum total of pleasures 
because ttjere are quantitative as well as quantitative differ- 
ences among pleasures. For both Bentham and Mill preferable 
actions and practices are those contributing to the living of a 
life filled with intrinsic goods.- utilitarianism myst assume that 
measured quantities of the good are quantities of pleasure of of 
happiness, and these quantities can be objectively measured. A 
basic problem for utilitarianism is that so many different answers 
have been given to the question* "In virtue of what are states of 
affairs good?" is it pleasure, happiness, or something else that 
makes soma states of affairs preferable to* others? 

The Appeal to Consequences. The Appeal of Consequences v. 

The principle of right is that an action is right if and only 
if it produces the greatest balance of good over bad consequences. 
These good consequences are located in states of affairs . Three 
basic criteria are propose^ by classical utilitarianism as 
criteria for judging actions: the factors to be considered are 
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the purity, the fecundity, and the certainty of consequences. We 
ace concerned here with the relation between actions and their 
actual, not merely their intended, outcomes. 

The concept of putity refers to intend as opposed to 
unintended consequencea. We would be checking for side effects. 
An action mixed with undesirable and unintended consequences is 
not as right as onf which will actually tend to produce purely 
desijrable consequences. In any social action we may expect 
undesirable consequences; fcad consequences are often foreseeable. 
These bad consequences must be weighed together with the good *hey 
accompany. Thus, it is not just any action which produces the 
greatest total good but rather the greatest net good when the bad 
is assessed. In extreme situations in which we confront only a 
range of undesirable alternatives, we must try to minimize the 
bad consequences, under the criterion of purity, as a way to 
decide the purest of actions from among the bad lot. The concept 
of purity calls our attention to a feature of utilitarianism which 
is a* concern with a range of alternative actions and practices. 
We keep our options open. We compare alternatives and assess 
their worth. 

Fecundity is the second concept we use to evaluate actions. 
Some immediate consequences are more desirable than others oecause 
they help cause desirable consequences later <j(ti. We judge a 
present action as helpful or hindering, to actions we will bave to 
take subsequentlyi Educative events are often judged for their „ 
"fertility in creating further good consequences. • Learning to read 
leads to reading, which can lead to further knowledge, and so 
forth. 

The certainty of actions producing good consequences is the 
third concept used in evaluating actions. Prediction of conse- 
quences of actions can be made with degrees of probability. Those 
with the~greater certainty are assigned greater weights. 'Typi- 
cally, short run consequences are more certain. Remoter 
consequences suffer the intervention of unforeseen factors. 

Purity, fecundity, and certainty are concepts we use to think 
about the connection between actions and consequences. We are m 
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evaluating actions on the basis of this connection. We must next 
look at four concepts used to analyze states of affairs. These 
four concepts are intensity, duration, propinquity, and extent. 

Intensity refers to estimates of how much g^od/bad exists for 
every affected person. Remember classical utilitarianism seeks 
the greatest good for the greatest number of all affected, with 
each person ^ounting equally. If we had a pleasure meter or a pain 
indicator we would be able to quantify the intensity of the 
pleasures and pains. A laugh meter or applause indicator can be 
taken as a way to measure intensity in states of affairs. 

Duration of a good or bad is another concept, in this case we 
need something like a pleasure or pain clock, measuring how long a 
given quality in states of affairs actually lasts. Long lasting 
qualities are preferable to short-lived ones. 

Propinquity is a third concept used in evaluating consequences. 
This judgment is also time-related. We are estimating how soon or 
how far away some good may be. We might ask how long is the in- 
terval of time between the present and the good or bad that comes 
about. This concept is very important "in cost-benefit analysis. 
If we think of costs and benefits as monetary, we can judge that 
if the benefit comes sooner thai! the costs, then we can increase 
the value of the benefit by the interest we can gain before costs 
are paid. But notice als<J a person who gives up a job to study 
pays an "opportunity cost" (i.e., the money he would have pade op. t 
the job) before any benefits accrue from study. In cost-benefit 
analysis, we subtract costs from benefits to define the marginal 
utility of the action. Economic utilitarianism can assign units 
of value (dollars) and, therefore , increase the precision of 
calculation. Measures of pleasure or pain, happiness or unhappi- 
ness are not as precisely quantifiable as costs and benefits. 
.Economic profit and loss are not, of course, the whole story of 
the goodness and badness of consequences of social actions. No 
single procedure for evaluating consequences is likely to be 
satisfactory. 
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Extent of good or bad consequences is the fourth concept. How 
many persons are involved in the state of affairs created by the 
social action? Wc are taking census measures. Clearly for utili- 
tarianism the greatest number is better than any other. Since 
each person is of equal worth, the more the merrier. In educa- 
tional evaluation, che total number of students affected b> an 
educational practice is a frequently used estimate of a program's 
worth. Evaluations are about consequences to individuals. Every 
individual merits apptaisal. 

in general, utilitarianism is a theory of social evaluation. 
Evaluation is seen as an objective means for making value judg- 
ments. These judgments may be about personal actions or social 
practices. Agreement among persons is desirable and such agreement 
is more likely to be achieved if value judgments are objective. 
Objectivity increases when judgments are quantified. Notice the 
assumption that quantification represents standards of what is 
thought ujiproblematic in value judgments. Quantification proce- 
dures may be, and often are, just as problematic as any other 
procedures; much depends on the events and objects being 
evaluated. See Concept Kap, "Utilitarianism. ■ 
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CONCEPT MAP 
UTILITARIANISM 



The Right 



action* (parson*!) > 
practices (social) 
causing tha good 



what govams 
tha right 



Principle of Right 



Arviction or practice ia 
rtght if and only if it 
prcducaa good conaaquancaa 



which may or 
judgad by three 1 
criteria 



purity 



fecundity 



cartainty 



, . i. a., no sida affects 
, . . i.e., creativity 
. . . i.e. probability 



The Good 



intrinaically valuable 
states of affairs 



for example. 

1 



I 



pleasure 



-L_ 



happineaa 



liberty 



health 



State* of Affairs 



the good consequences of 
right action 



intensity 



durst ton 



| propinquity^ 
| extent | 



meaeured b t 

e.g.. a pleasure or pain meter 

e.g., a pleasure or pam closk 

e.g.. a measure of immediacy 

e.g.. censua measurer 
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Review and Summary 

Evaluation documents present multiple claims. Of all the 
claims, four are important: factual, valuational, conceptual, 
and methodological. Factual claims are answers to questions about 
what exists as an object or event. Pacts, that is, records of 
events, are the facts of the case and are important to establish 
descriptively. But they must be transformed into data, that is, 
given some order and significance. Data then are used to back 
knowledge claims. Value claims are answers to questions about 
the existence of value. Simple claims of value are transformed 
into five fundamental forms of value claims. Conceptual claims 
are answers to questions about the meaning of concepts. The 
meaning of concepts is a function of their use in a context of 
inquiry and evaluation. We have developed the technique of 
concept mapping as a way to make conceptual structures evident. 

We are also making methodological claims. Actually, this 
document so far presents a method of analysis for the structure 
of claims. Methodological claims are answers to questions of how 
we establish other claims. QUEMAC Value is a method of analysis. 
Concept mapping is a technique of this method, using the 
heuristic of the Vee is also a technique of analysis. 

To know what multiple claims exist in evaluation studies 
should be helpful. It should be helpful in the analysis of 
studies as well as in the creation of new studies. It is from an 
analysis of prior studies that we derive criteria of excellence. 
These criteria can be used to judge existing studies and to guide 
the development of new studies. 

Utilitarianism, a version of the classical view, is presented 
as a theory of evaluation. It proposed that we evaluate the 
consequences of social actions and practices. It proposed several 
concepts (purity, fecundity, certainty, intensity, duration, 
propinquity, extent) and techniques of the evaluation of social 
actions. 
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What is Educational Value? 



We come into the world through birth, but we come into 
possession of our powers only through education. Education is 
that process by which we come into possession of our powers for 
the exercise of intellect, emotion, imagination, judgment, and 
action. It is also a process by wbich we come to self- 
understanding, including the capacity to "change our minds". If 
any object, program, pattern of instruction, and so forth has 
educational value, then its value resides in its utility for 
education. That is to say, its educational value is its utility 
for nelping us come into possession of our powers. Hence, to say 
that one educational program or object is educationally better 
than another is to say that it has more educational value. Iz 
contributes more to our coming into possession of our powers as 
human beings. 

Cf course, any educational program, school, or other object 
of evaluation may have other kinds of worth than educational 
vortij. It may, for example, have social value as well as educa- 
tional value. That is to say, a particular object of evaluation 
may nave littie efficacy in helping us to come into possession of 
our powers and at the same time have great value in helping us to 
come into possession of a job or in helping us to attain a 
particular position in life. Coming into possession of these 
things is also something of value, but their value is 
distinguishable from educational value. 

Among the many activities of formal education, some will have 
more worth than others. And among those that have worth, some 
may have great social utility but little educational utility. 
And conversely, it may happen that among the activities for formal 
education, some may have little social utility and, at the same 
time, be of surpassing educational worth. 

Every experienced teacher has witnessed from time to time what 
happens when a student finds just the right word, has mastered 
the new technique, grasped that fundamental principle, sharpened 
the ear to hear just the right note, or obtained the perfect 
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result from a labored experiment. On such occasions of 
significant learning, the usual response of human Deings is to 
laugh. Learning, when it is truly educative, is always an 
experience that borders on hilarity. If someone were to happen 
upon the scene at such a time and ask, "But of what use is it?" 
the proper response would not be to answer the question by some 
contrived account of utility, but to reject the question alto- 
gether. Anyone who has to ask at such moment* "Is it good?" "Of 
what use is it?" just doesn't understand what has been going on. 

The appearance of such hilarity or its accompanying subdued 
chuckle is a sign of an experience of educational worth. It -would- 
remain a sign of the presence of educational value even if the 
thing learned or grasped lacked any social utility at all. If we 
could discover some combination of text, teacher, and pattern of 
question and answer thdft would invariably produce such results, 
then we would have discovered an educational object of surpassing 
educational^WC^th if it had no other kind of value at all*. 

Such events' have educational value ^ot only because they 
reveal what it is to come into possession of our powers, but alfb 
because they permit us to come into possession of our world, the 
world we live in. And they have this significant value because 
such events consist in the construction of meaning. 

We understand the creation of meaning out of human experience 
when we understand that one something (an A) can come to stand 
for another something (a B) . The footprint in the sand is taken 
as a signjd>at a person probably walked there. The footprint that 
is present is a sign of the person who is absent. The footprint 
is a record of an event. If the ocean washes over the footprint 
that record will be destroyed. There is then no remaining basis 
for a sign of those past events. We say that smoke is a sign of 
fire, and sometimes it is. We say that dark clouds are a sign of 
rain, and sometimes the rains will come. When we take something 
that is in present time (the footprint) to be a /?ign of something 
else past (the person), then we are making inferences. We are 
also making meaning. When we say that "A is the sign of B H or "A 
stands for B", we are also saying "L means B". The footprint 
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becomes a uign. It stands for or means a person. The smoke means 
that fire is likely; the clouds mean rain. 

Social meaning is an achievement of shared human activity such 
that the sane sign is taken to stand for the same event for 
different persons and at different times. Meaning is generated 
out of share* experience. This possibility of shared meanings so 
tbwt different persons can undergo the same experience is what 
makes educating possible. Meaning is social. When meanings are 
constructed, our powers as human beings are roused. We come into 
their possession. But when meanings are constructed r we also come 
into possession of our world. Meanings connect things. And it 
is this feature also that gives certain events their educational 
value. The construction of meanings connects the present to the 
past — the footprint to a person. They connect the present to the 
future— the clouds to the likelihood of rain. They connect events 
to causes both present and future. They 'connect also facts to 
principles and hope to memory, in these kinds of events, arising 
out of the construction of meanings, we discern not only our 
coming into possession of our powers but also our coming into 
possession of the world we live in. Educational value arises out 
of the construction of meanings that tie things together and *:hus 
create our world. 

It follows that not everything learned has educational worth, 
because not everything that is learned has utility for making 
connections, for constructing meaning, and, therefore, for 
arousing and perfecting our human powers. Learning some "things 
will have large social value and little educational value. 
Consider learning to tie a bowline or a half-hitch. These are 
very nearly the seaman's universal knots. The widespread acqui- 
sition of such skills has a large social value for a seafaring 
people. But in saying this we leave it still an open question as 
to whether learning to tie a bowline and a half-hitch has 
educational worth. Does the acquisition of that skill make 
meanings? These are useful knots in trimming the sails and in 
mooring the boat, but do they make connections or meanings — signs 
to events, past to future, hope to memory? This, we do not yet 
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know. The social value of learning such skills is clear. Their 
educational value is not so clear. Their mastery makes for 
security in a storm. Does their mastery also make meanings, 
connections of another kind? 

It is important to note also that in this account of 
educational worth, the educational value of any object (curricu- - 
lum, lesson, pedagogy, text, school) is ne\er to be discovered 
exclusively in the object itself. Educational worth will almost 
never be discovered entirely ill "what happens" to the learner as 
a result of undergoing the lesson. It will be discovered only in 
what the learner does . Hence, the same lesson may have large 
educational value for one learner and little for another. Which 
it is depends upon whether the learner makes connections, 
constructs meanings, comes to grasp the meaning and whether, 
therefore, that hilarious experience of confronting educational 
worth occurs. 

What is in the lesson, its contrivance, its placement in a 
sequence, the method of approaching it — all of this may be 
predetermined. But the educational worth of what is learned can 
never be predetermined because it must remain always an open 
question as to what the learner will do. What is being taught 
may be predetermined; but what is learned and whether what is 
learned has educational worth, must remain always an open , 
question, since it depends upon what the learner does . The, study 
of "effects" is destined always to avoid the measure of educa- 
tional worth because what needs examination is not what the 
lesson does, but what the learner does. 

We think there is probably an inverse relation between our 
capacity to preplan what the learner will learn and our capacity 
to measure the educational worth of what is learned by measuring 
our preplanned objectives, in other words, we can control for 
what is learned in any lesson (curriculum, text, pedagogy, school) 
only to the extent that we can control what the learner does with 
the lesson. The educational worth of the lesson resides, however, 
not in the lesson itself — whatever its design — but in the 
combination of the lesson and what th* lesson does with it. 
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The educational value of any object resides in its utility 
for assisting us in coming into possession of our powers and 
coming into possession of the world we inhabit. The possibility 
of educational worth, like the possibility of education itself, 
rests upon the fact that meanings are social constructions that, 
on the one nand allow us to exercise the powers of inference, 
self-understinding and thoughtful action, and, on the other hand, 
tie things together in the world that we inhabit. 

Grasping the weaning, especially of materials that embody the 
criterie of excellence, is also fundamental to the educative 
process and to the grounds upon which educational value rests. 
The occurrence of grasped meaning is often expressed in statements 
such as: "He has the sense of it." "She got it, by golly, she 
got it." "He has the hang of it." "Now I see what you mean." 
"So that's the point, eh?" "If you just see this, then all the 
rest follows." "You mean Caliban, Ariel, and Prospero stand for 
the tripartite soul of Renaissance Man] l m Grasping tne meaning 
is something that each of us must do; it is not part of what the 
lesson contains. We must do it ourselves even though we may need 
extensive help from others. Learning, that is, the kind of 
learning that constitutes learning the materials to the point of 
mastery, is the responsibility of an individual. It is not some- 
thing that can be shared. When learning really requires mastery, 
that is to say, when it really has educational worth, it requires 
the grasp of meaning, indeed, when learning has educational 
worth, it is the grasped meaning that one initially learns. 
However, given the grasped meaning, then one's experience is 
different, and additional learning becomes possible. Meaning is 
extended; connections are made. 

Consider, in order to learn to tell a joke we must first "get 
the point" of the joke. If we do not first "get the point", then 
we cannot learn to tell the joke; but having "got the point", we 
can. The example is fruitful, consider the dismay that is felt 
when after telling a joke someone says, "I don't get it:" To 
explain the joke by making the argument of the joke explicit 
always destroys the joke* We must "get the point" but we cannot 
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be "given the point-. Getting the^int ia the first and 
necessary step. Then we can learn to tell the joke. " (Note 
"getting it" in the case of jokes always requires making an 
inference, even in the case of sight jokes.) in such cases, and 
indeed probably in all cases, what we learn first is "the point". 
The grasped meaning is learned first. What follows is the 
possibility of new learning, learning tQ tell the joke. 

For a seafaring people it is easy to get the point of being 
able to tie a bowline. That skill has immediate and visible 
social value. It is needed in order to trim the sails. So 
learning to trim the sails has its point, and it is not difficult 
"to get". But leaning when to trim the sails — that is more 
difficult. It is also of greater educational worth; not, however, 
because it is more difficult, but because it is more fruitful, 
more fructifying, more pregnant with inferences, more filled with 
potential meaning. It evokes more connections in the arousax of 
human powers and is more powerful in putting together the world 
in which a seafaring pecple live. But whether we are concerned 
with learning to tie the bowline or learning when to trim the 
sails, the first step is "getting the point". Grasping uhe 
meaning is the prelude to learning. It is the Jtind of learning 
that permits learning to continue. Where there is no point, no 
meaning to grasp, there is little probability that anything of 
educational worth will occur. Yet, this "getting the po>nt" is 
inescapably an act of the learner, not a property of the lesson. 
What has educational worth may be discoverable in the intentions 
of the teacher and in the plan of the lesson, the curriculum or 
the program, but it will be discoverably only in the actions of 
the learners. Therefore, the fact that everyone taking the lesson 
has learned to tie the bowline, does not establish that the lesson 
had educational v»lue. Nor does it establish that the lesson was 
educationally better than one in which fewer learned to tie the 
bowline. Such evidence establishes that the lesson was better in 
getting persons to possess the ability to tie the bowline; but it 
leaves untouched the question as to whether the lesson had greater 
or less educations lfl rorth. What is taught by the teacher and what 
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is intended by the lesson is surely never all that is learned by 
the student. 

To teach then, is to try deliberately to change the meaning 
of students 1 experience i and students must grasp the meaning, they 
must "get the point - , before learning, deliberate learning can 
occur. Often in the context of educating; a teacher may make a 
point and find that the student response is laughter. That is 
one way that skilled lecturers tell whether their audience is with 
them on a point of real significance. Nor is laughter the only 
index of the occurrence of grasped meaning and the only link 
between the powers of intellect and the capacity for emotion. 
Anger, fear, shame, affection, and desiring are also indices of 
the feelings that accompany thinking when thinking is truly 
educative. Educational value is evident in those moments when 
grasping the meaning and feeling the significance of that meaning 
come together . When cognition is educative, then it is never 
separaole from emotion. When cognition occurs without emotion, 
then it is always cognition that does not matter. # It is^learning 
and knowing that is not truly educative. 



Political Paradoxes of Unanswered Claims 

It may be worth observing that we live in a time when there 
is a rising complaint that the schools are less excellent than 
they can be, -less effective than they should be. There is a loss 
of confidence. If we are correct in our portrayal of educational 
value, then there is a notable irony and paradox in this 
complaint. 

For if we are correct, then it follows that nobody is in sny 
doubt as to the nature of educational value. Everyone knows what 
it is. There is no privileged access to the idea. It is not an 
occurrence known to philosophers anymore than it is known to 
parents. Educational worth is known, discoverable and recognized 
by everyone. Let us suppose that the public complaint is to be 
interpreted as the claim that the schools do not educate. What 
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they do, on the whole, does not have educative value. But let us 
suppose further that, the response of school administrators, 
legislators, and teachers is to do whatever is needed to raise 
the achievement scores of students on standardized measures. It 
is an important possibility that the answer given, though an 
answer to some quest ion , may not be an answer to the complaint 
being raised by the general public. It is asked, "Why don't the 
schools do a better job of educating?" to which the response is 
"Let us raise the levels of achievement." But levels of achieve- 
ment may be raised without improving the level of educational 
worth. We do not mean that raising levels oS^achievement is a 
bad thing to do. We rntun only that it may not Dte an answer to 
the complaint bein^ raised. The question asked i^ not the 
question answered, and the answer given is not an answer to the 
question asked. 



Value and Having Values 

Whether we are teachers, parents, or members of the general 
public, we want to *8k "What values do our children have?" "What 
values should *iey have?" Everyone recognizes that these are 
important questions in the conduct of education. The" may also 
be important questions for the conduct of educational evaluation. 
But few are prepared to answer them with any confidence. We do 
not propose to answer them either. But, in this section, we do 
aim-to examine these questions through their different permuta- 
tions and logical forms to study their meaning, their implications 
for practice, and the logical requirements tfrat would have to be 
satisfied by any attempt to provide answers. We do propose to 
consider how the nature of these questions may influence the 
practice of evaluation. 

"What values do our children have?" "What values should they 
have?" These questions are by no means clear, in fact, as we 
shall try to show in a moment, it is not clear what intelligible 
sense, if any, can be attached to such questions, although we 
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have no trouble recognizing that they occur with frequency and 
are of great concern, to many people. What do they mean? What 
would count as answers to them? 

In general, we may discern in our own tradition of ethical 
reflection that claims about valuing, about worth, and about the 
characteristics of persons, are different kinds of claims. 
Consider the following classification: 

Table 1 

Language of Value Assertions 
Classified 



Type #1 

"Value" as Verb 



Type #2 
"Value" as Predicate 



Type #3 - 
"Value" as Virtue 



A 


values 


honesty. 


Honesty has value. 


A is an honest 










person. 


A 


values 


freedom. 


Freedom is valuable. 


A j.s a free 










person. 


A 


values 


learning. 


Learning is a worth- 


A is always ask- 








' while activity. 


ing questions, 










etc. 



What is the conceptual principle that generates this 
classification? It is easy to see that in the first column, the 
word 'value" occurs as a verb. Valuing is an activity of persons. 
In the second column, ^wever, it occurs as a predicate, a prop- 
erty of things other than persons. The first sentence of type #1 
is a statement about a certain individual, A. But the corre- 
sponding statement of type #2 is a statement ?bout honesty. In 
the third column we should note that the term "value" does not x 
occur at all. Neither do any of Its cognates— -"valuing", 
"valuable 11 , "worth", "worthwhile", etc. instead, what was a verb 
in statements of type #1 and a predicate in statements of type #2, 
is now replaced by a complex predicate describing some aspect of 
a person's character or personality, and the concept of "value" 
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does not appear at* all. Expressions o« type #3,, like those of 
type #1 are about some individual person.* 

Clearly, these are assertion* of three distinct 16qlcdt. 
types— "verb statements" , "predicate statements", and "virtue 
statements". Let us return, however, to our original question, 
"What values do our children have?" What would count as an answer 
to this question? TO which logical type would the answer belong? 
Are we asking a question of type #1— "What do our children value ? * 
or are we asking a question of type #3~"What kinds of persons 
are our children?" Clearly, our question is not of type #2. It 
is not a questibn about what things have value. 

It should be apparent then that when we ask "What values do t \ 
our children have?" our questibn gets translated into something 
like this: 

(i) What kinds of things do they valued or^ 
(ii) What beliefs, attitudes, habits, or affections do o"'r 
children have? What kinds of persons are they? 
These are perfectly legitimate questions to ask. We could even 
add another: 

(iii) How can they come to have different values? or "How 
f can we lead them. to have different values? 

If we assume that our concern with value and with values in 

education arises, from our desire to shape the values of children, 

then we sihalJ want to ask this last question. Indeed, it. is the 

question toward which all others lead. 

However, it is important — perhaps even of overriding 

importance— to note that if we confine our attention to claims of 

type #1 and type #3, then we shall very quickly run up against a 

barrier. There will be important questions that we cannot ask. 

For example, provided we limit attention to claims of type #1 and 

#3, then we will be able to ask how our children may come to have 



*We shall leave it an open question* for the moment, as to 
whether statements of type #3 can be translated without remainder 
into statements of type #2 and vice versa. Upon the answer to 
this question several different theories -of moral development may 
be distinguished. 
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different values than they presently have, but we shall be unable 
to ask 

(iv) How can they come to have good values? 

This is the inescapable question! But the difficulty is that 
in asking it we are now asking a question not about what values 
people have, but about what values have worth* We are now asking, 
whether we like it or not, a question that deals not with What 
people value, or with what kinds of persons our children are. We * 
are asking a question about wha t t things have wprth. In short, 
this last question is both unavoidable and alSQ unavoidably a 
question of type #2. 

We need to be able to ask questions about what things have 
worth and what things have more worth than others. If we lived 
in a world where such questions cannot be asked, then we would * 
live in a world where some of the most important educational 
questions also could not be asked or could be asked only in very 
dangerous* ways. Depending upon whether we admit such questions 
to our educational language, whole different educational worlds 
will emerge. And even if we do admit $uch questions, different 
educational worlds will emerge depending upon how we define them. 

There was a time when the roost basic of all educational 
questions was constructed by asking how we can educate so that 
persons will valine' those things that have worth. But if it is 
impossible to ask what things have worth independently of what 
people value , then it is impossible to ask the educational 
question in this way. An entirely different educational world 
comes into existence when we rule out questions of type #2 fronr 
our educational discourse. 

Nevertheless, in a world where questions of that type are 
inadmissible, e can still ask what people value, what k£h$a of 
persons they are and how they can be led to have other values than 
they already have. And in that kind of world, we would be able 
to ask these questions, not only about saints and sinners of the 
ordinary kind, but also Nazis, tyrants, barbarians and criminals 
of all kinds. The trouble is that in such a world we would be 




unable to say anything about whose values are better or whether 
any of their values are good. We could speak only of what they 
value and what kinds of persons they are. 

In such a world, the question as to what things have worth 
would have to be answered by asking "... according to whose 
values?" We would have to settle upon a group of people and 
suggest that good values, even educational values, are whatever 
they value. Should it be the values of whites, businessmen, 
teachers, scholars, churchman? Should it be the dominant values 
of the society? Could we determine what any of these are? But 
suppose we could decide. The question would remain, who should 
decide? In the kind of educational world we have been describing, 
the kinrl of world in which questions of type #2 are not admitted 
to educational discourse, the chief feature will be the presence 
of deep and seious problems of authority. The important point to 
observe in this pattern of thought is that it represents not a 
way of a nswering questions of type #2, but a way of avoiding the 
need to confront them. It is not another way of raising questions 
of type #2. It is only what will follow from their consistent 
evasion. 

In this account we have meant to characterize the present 
educational world. The fact that the problems we have described 
are the problems that we confront can be taken as evidence that 
the world we live in is one in which instead of speaking of what 
has worth or value, we can speak only of what values people have . 
In this kind of world about the best that evaluation can do is to 
survey persons' preferences on matters of worth. In the practice 
of evaluation we can survey :he attitudes of persons, their 
opinions, dispositions, and seek means to change them, or lacking 
that, just rex>rt them. But if, in addition, there is the educa- 
tional task of change, then we have problems of authority, power, 
who decides, and so forth. Education, in that kind of world, is 
likely to turn out either to be therapy, manipulation, indoctrin- 
ation, or simply a blatant exercise of power and compulsion. All 
of these characterizations have been defended as true in current 
educational criticism. 
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But the central point can be discerned in a single example. 
The statement 

(v) A values his education 

does not mean the same as 

(vi) A's education is valuable, excellent, thorough, etc. 
(v) is a type #1 statement, and (vi) is a type #2 
statement. But in a world in which type #2 statements 
are inadmissible, the only reading we can give to (vi) 
is 

(vii) A's education is valuable to him 

which means simply that he values it. In short, in such a world, 
(vi) means the same as (v) . That is clearly false. In fact, to 
suppose that "A values his education" means "A's education is 
valuable" is not simply false, but absurd. Yet, that is the 
consequence of a world in which questions of type #2 are not 
permitted to enter and, therefore, are not permitted to enter into 
the practice of evaluation. We are left with an educational world 
in which evaluation is limited to surveying people's values and 
littU else. "How many like twinkies?" 

How co aid such a state of affairs arise? We are making here 
a difficult point. The central thesis is that in the modern 
world, the language, therefore the concepts of value or worth, 
have been transformed by cultural forces and by philosophical 
amnesia. Can we trace the origins of that transformatibn? 

The notion that people have values, though commonplace in our 
world, is a distinctly modern and highly ambiguous thought. It 
is an idfca without a h/.story. The idea that things have value 
(type #2) or that people value things (type #1) is, of course, 
very old, and we do not intend to deny these elementary facts. 
Expressions of the form "X has value", "X is worthless", or "X is 
valuable" have always played a central role in Western thought. 
But th*s usage takes the concept of "value" to be a predicate of 
objects or a pr ^>erty of relations. ^ 

We could say that the central problem of classical moral 
thocght was to give an account of what constitutes the good life— 
what really has worth or value, and how those goods can be 
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embodied in human life, subject as it is to luck and misfortune. 
But none of that discussion would make any sense at all were it 
not for the fact that people value pleasure, wealth, power, 
security, friendship, intelligence, and so forth. But this usage 
takes the concept of "value" to be a verb . Valuing is something 
that human beings do. It is an activity, not something that they 
have. In our tradition it has always made sense to speak of 
persons valuing different things (verb), and to speak also of what 
things have more worth than others (predicate) . On such a view 
it was possible to state the fundamental problem of valua in 
education by asking how it is that we can bring people to value 
(verb) those things that have worth (predicate). 

But these familiar and basic ways of thinking about value do 
not conform at all to the modern notion of values (plural noun) 
as a possession of persons. When we speak of people having values 
we are using a cognate to a verb or a predicate in a way that 
corresponds neither to its use as a verb nor as a predicate. It 
is a usage that speaks of the state of human being, not an 
activity. It speak'' of the condition of a human being not the 
worth of anything presented to that person. It is a distinctly, 
modern idea. It would, in fact, make no sense to a classical 
thinker to suggest that people have values at all. They may have 
virtues, knowledge, beliefs, traits (type #3), luck, wealth, 
pleasure, honor, happiness, or certain dispositions. That they 
might have values in addition to these, or in contrast to them, 
is something that just cannot be s?id from such a point of view. 
Such a thought can be expressed only from a distinctly modi rn 
point of view. 

One might advance the view that until early in this century, 
nobody had values. What they had were beliefs, dispositions, 
habits, desires, inclinations, sometimes good fortune, and even 
from time to time a little bit of luck and happiness. It is 
instructive perhaps to note that at least until the late 
nineteenth century, no major moral or social thinker in the 
Western world used the concept of "values" (plural noun) as it is 
used in modern social science or social commentary or evaluation? 
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This fact is likely to strike the modern consciousness as odd in 
the extreme. It is a claim that needs some explanation. How did 
this conceptual claim develop and what are its consequences for 
education and for educational evaluation. 

Having values and the Concept of Marginal utility 

One way to grasp how the idea came into the world that people 
have values is to attend to the powerful emergence in the past 
one-hundred years of the concept of marginal utility. 

The essentials of such a concept are easy to grasp. The 
consequences are not so easy to grasp. It begins in the notion 
that value is utility, and that utility of a commodity or anything 
is something inherent in that thing. The value of water lies in 
its utility. It resides in the capacity of water to quench 
thirst, to wash, cleanse and so forth. But the utility of water, 
its value, is something that resides in the thing itself. It is 
by virtue of certain natural properties of water that it has such 
utility. It would have such utility even though there were no 
persons using it. The utility of water arises from its natural 
properties. Those properties are the roots of its value. They 
constitute, as it were, the use-value of water and they are 
implicit in the character of the thing itself. That is one view. 

But, of course, there are circumstances in which water is 
scarce and others in which it is abundant. Where it is scarce 
some persons who have it may ask others who want it to pay a 
price. They may ask something in exchenge for the acquisition of 
the use value of water. In short, they may ask a price for it. 
Thus, water has not only a use value, a worth implicit in its very 
nature, but it also may have exchange value, and tne exchange 
value is whatever a unit of water is able to secure in exchange 
for some other goods. 

Now if we view the matter in this way, then a question arises. 
In addition to its use value, a commodity has an exchange value. 
The exchange value is its price. How can we determine a fair 
price for any good? Clearly, a "fair price" should be related to 



use value, i.e., inherent value. We might say that the exchange 
value of any commodity , any exchangeable thing in the world, 
should be related in some way to its utility, its use value. But 
how? 

Economists in the early nineteenth century struggled witii this 
problem and came up with essentially two answers. The first is 
that the exchange value of a good should be determined by calcu- 
lating the number of "utils" (amount of utility) that constitutes 
its inherent worth. That is to say, a sweater in a cold climate 
has more utils than a sweater in a temperate climate. Having more 
utils, it has more value, and, therefore, it can and should 
command a greater exchange value or price. The second view vas 
that the price of a commodity is a function of the effort needed 
to produce it. Hence, value is viewed not as some natural prop- 
erty of utility embodied in the object, but as the amount of labor 
needed to produce it. Hence, the labor theory of value. 

This problem of relating price to use value or utility was 
examined, twisted, and wrung dry without notable advancement from 
Adam Smith "to Karl Marx. But with the neo-classical economists, 
and especially in the work of Jevons and Marshall, the problem 
was given a substantially different formulation in the theory of 
"the final degree of utility". What is that view? 

We shall give the idea two, logically equivalent definitions. 
Here is the first. We are unlikely to find anything surprising 
in the claim that a person who has an entire garage full of bread 
is unlikely to exercise great concern over getting one more. But 
neither would we be surprised that a person who has none at all 
is likely to place a great deal of importance on getting one. 
Hence, the next loaf of bread is likely to be valued highly by 
the one and not highly at all by the other. Indeed, we would find 
nothing surprising in the claim that a person who has none and, 
therefore, values the next loaf highly, is likely to *alue the 
next loaf less highly after he manages to store a few away. 
Hence, the value of a loaf of bread, or any other good, is a 
function of the stock of that good that one already has. The 
marginal utility of any good then is always defined as the value 
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of the next unit of some commodity in relation to the stock of it 
that one already possesses. The next five dollars is likely to 
be vslued highly by a person who has few and is likely tq be 
viewed as a mattet of frivolity to a person who already has 
millions. Its value is its marginal utility, not its inherent 
utility, and it tends always to be determined by the amount of 
that good that one already possesses. 

But there is a second way of defining marginal utility. We 
can say that the marginal utility of any good is the amount of 
some other good that one is willing to exchange for the next 
increment. To say that the value of the next incremental loaf is 
slight for the person who already has a thousand is to say that 
that person is unlikely to give up much of anything in order to 
get the next loaf. And to say that the next loaf has larger 
marginal utility for the person who has none is to say that that 
person is likely to give up a great deal of some other good in 
order to get ti\e next incremental loaf. Thus, we may produce a 
logically equivalent definition of marginal utility by noting that 
the marginal utility of anything is the amount of some other good 
that one is willing and able to give up, for the next increment. 
Marginal utility then, j.s the rate of exchange. 

Clearly, under these definitions* by introducing the idea of 
marginal utility we have been able to see that the value of any 
object i£ its price. Price now becomes the measure of value. 
Use value and exchange value come to be united. 

But to see the beauty of this transformation and it* 
importance for problems of evaluation, we need to take just one 
more step. It is important to note that the concept of marginal 
utility is always defined as utility to someone or to some aggre- 
gate of individuals. It is not a property of any good or 
commodity in itself. It turns out, by this analysis of value, 
that there is no such thing as the inherent value of anything. 
There is only its value to some individual. If value is utility, 
and if utility is marginal, then value is always the value of X 
to some individual. Thus, we can purchase a powerful view of 
consumer behavior by giving up the view that things have value. 
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(We give up type #2 claims.) In doing so it becomes possible to 
speak instead of people having values, by which is meant simply 
that they have different individual calculations of marginal 
utility. 

One may be pardoned for entertaining the suspicion that the 
logic of such a view is impregnably circular. But such suspicion - 
can in no way detract from the significance of such a conceptual 
shift in the conception of value for the development of economics 
or from the claim that it is a pattern of thought indelibly 
stamped upon the culture of the modern world, and thus, upon the 
practice of evaluation. A world in which value is understood as 
marginal utility is a world in which nothing is presented to us 
as valuable in and of itself. It is a world in which it is 
increasingly difficult to speak of the worth of things. Instead, 
it becomes natural and easy to say that nothing has worth of 
itself, value is not something presented to us by our experience 
in the world. Rather, all we can spe*K of is the values or 
persons, by which is meant the fact that each of us has his or 
hew own calculation of marginal utility. Nothing has value except 
as it has value to this or that person. 

In such circumstances, we can no longer formulate the most 
fundamental educational question by asking how persons can be led 
to value (verb) those things that have worth (predicate) because 
nothing can be construed to have, worth except to some other 
individual. The most fundamental of all educatibnal questions 
now comes to be redefined. We must ask instead, ?How can we get 
people to have good (?) values?" and "Whose values are we to 
inclucate?" and "Who is to decide?" We have, then, problems of 
authority, difficulties with indoctrination and questions 
concerning by what right we insist on t':e power to H muck around" 
with children's preferences. 

Thus, we can understand why it is that to ask questions about 
people's values is not yet to address the question of what has 
worth. To transform the concept of marn'nal utility into a 
cultural presupposition would be to create the kind of world from 
which value, including educational value, has been evacuated. If 



55 

61 



we can no longer speak of what has value in such a world, only of 
what has marginal utility to this or that person or group of 
persons, if we are permitted only to speak of persons' values, 
then it should not surprise us that educational evaluation will 
have difficulty addressing the problem of evaluating the educa- 
tional worth of things. We would have to replace all such 
discourse by discpurse about what contributes to the formation 
and the alteration of persons 1 values. Besides creating a 
condition of crisis in educational authority, what other 
implications flow from such a view of value of having values? 

In order to see the prima facie inadequacy of such a state of 
affairs, one need only recall the familiar saying, "He knows the 
price of everything and the value of nothing." That is an 
accurate account of anyone who truly inhabits the limited world 
in which value is defined, as marginal utility. That would be a 
world in which statements of type #2 are denied entry into 
educational discourse. 



Another Value Question; The Virtue-Function Analysis 

In Part I (see page 18), we distinguished five different 
kinds of value claims and their corresponding questions. In this 
brief note we wish to suggest a sixth that may be both discrete 
from the others and useful in the context of evaluation. We shall 
refer to it as the virtue/function claim. 

6. The virtue/function claim 

Is A a good X? Is Head Start a good compensatory 

education program? 

If A is a good X, If Head Start is a good compensa- 

then by virtue of tory education program, then by 

what y, is A a virtue of what characteristic is 

goo4 X? it a good program of that kind? 

What are the virtues of a 
compensatory program? 

The impact of the virtue/function claim is to add to the array 
of questions useful in applying QUEMAC in analyzing an evaluation 
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document. The underlying question is this: "Is A a good one of 
its kind?" The effect of asking this question is first, t>at it 
forces us to classify the object of evaluation. We must be 
prepared to specify what kind of thing it is. But secondly, this 
question also obliges us to specify the characteristics that make 
the" object of evaluation a good representative of its kind. 

Thus, much depends upon whether we conceive of Head Start as a 
compensatory program, an early childhood program, or a program for 
child care. These are different kinds of things. Excellence of 
one kind may be different from excellence of another kind. We 
would say, from within the philosophical tradition, that whether 
Head Start is a good program of one kind would require a dif: -ent 
set of virtues that would be required if it is a good program of a 
different kind. 

But this reference to "virtue" deserves some explanation. The 
underlying idea stems from Greek. In the Greek language, the word 
that we slovenly translate as "virtue" is arete . In colloquial 
English, the* word "virtue" carries with it moral connotations that 
the word arete does not have. Arete means simply "excellence". 
Everything has its arete . The arete of a knife is to cut. But 
the arete of a surgeon's knife is to be small, light, and to cut 
cleanly with little pressure. A cleaver is also a knife, but its 
arete is heaviness, thickness, and balance, as well as sharpness. 
Their arete is different because their function is different. 
Arete , then, is excellence always in relation to function, which 
is to say, in relation to kind. 

The arete , that is the virtue, of a race horse is speed. But 
the arete of a quarter horse is the ability to attain full speed 
suddenly. The arete of a cutting horso is agility in all direc- 
tions, as well as a quite specifiable kind of intelligence. The 
arete of a draft horse is strength. Each time we specify a 
different kind, there will be a different set of virtues, a 
different arete , corresponding to that difference in kind. 

So there are two things to note about this underlying thought. 
The first is that che concept of "virtue"~or arete — is not a moral 
concept, even though it is a concept of value. It means simply 
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"excellence"; but secondly it means always "excellence in relation 
to function". If we ask whether A is a good X, and answer "Yes", 
then we must be prepared to say "by virtue of (literally) what Y 
is it a good X — that is to say, "by what virtues" is it a good X? 

A Concluding Note : 

Two points are important. First is the fact that the^virtue/ 
function claim is, at the same time, the iaost general and the most 
specific of all the value claims that we have distinguished. It 
is the most general in one respect because it has the widest range 
of application. The virtue/function value claim is relevant to 
evaluation of an educational program, pedagogy, plan r text, or 
even in reviewing a book, It is a central value question for 
^valuations and evaluation reports, and even for the evaluation 
of evalu&tors themselves. In such case we can ask "Is A a good 
X?" 

But the virtue/function set of questions will also be decisive 
in'determining which of the other kinds of value claims will be 
relevant in examining an evaluation document. That is to say, 
when we respond to the virtue/function value question, we will be 
determining whether, from that point onward, the relevant problems 
are the claims of instrumental value, intrinsic value, comparative 
value, decision, or idealization. In short, the virtue/function 
value claim is the one that determines which of the other kinds 
of value claims are most relevant to making the philosophy of the 
evaluation document explicit. 

But the virtue/function claim, aside from these powerful 
generalities, is also the most specific of the value claims. It 
is the most specific because it is the one that requires the most 
detailed specification. It requires us to identify the virtues 
that are relevant, and that specification almost always requires 
a concrete level of illustration and reflection. It requires 
detailed philosophical analysis, in other words. < 

But there is a second point to note* Both in discussing the 
nature of educational value or worth and in exploring the meaning 



58 




of "having values" it may seem that we have been advancing the 
claim that value is relative. And that would be a correct 
impression. The nature of the virtue/function value claim seems 
to reinforce that view. But it is equally important to note that 
at the same time that we have been advancing the view that value 
is relative, we have been arguing against the view that value is 
subjective. 

The virtue/function value claim should help us to see this, 
point. Arete is always excellence in relation to function. But 
nobody should confuse this claim with the very different assertion 
that value (or excellence, virtue) is subjective, or that value 
is to be understood simply as an expression of the subjectively 
different marginal utility schedules (preferences, dispositions, 
inclinations) of individuals. Whether any given educational 
object has educational worth is a fact that is relative to 
individuals. Nevertheless, educational value is objectively 
present or absent, relevant or irrelevant. Virtue is relative, 
but not subjective. It is relative to individuals, but not an 
expression of preferences, likes, or aversions. 

Yet, in another sense, virtue (read "excellence" or "value") 
is always the same. It is always excellence in relation to 
function. We may ask "What is a good evaluator?" The answer will 
be that a good evaluator is one who exercises the virtues of an 
evaluator. We may ask "What is a good historian?" The answer is 
that a good historian is one who exercises the virtues of an 
historian. In a precisely similar way we can ask "What is a good 
human being?" The answer, though more general and more difficult, 
is no more subjective than the others. It is that a good human 
being is one who exercises the virtues of being human, i.e., those 
excellences that are appropriate to a good representative of that 
kind. The answer is to be found, then, in the study of a subject 
called anthropology — the study of human- kind. It follows that if 
the study of history has educative value, then, by the virtue/ 
function question, it has educative value because it assists us 
in acquiring a memory— which is one of the powers inherent in what 
it means tp be a good representative of a certain kind, viz., 
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huaan-kind. That is to say, if A is a good X (where X is human- 
kind) , then A is a good X partly in virtue of the fact that A has 
a social memory, because having such a memory is one of the human 
virtues. This claim may be true* It may be false. But 4 there is 
nothing, subjective about it. Although there is something relative 
to the individual involved in the claim that any particular path 
is a good way toward the acquisition or cultivation of that 
virtue, nevertheless, 'there is nothing subjective about the claim. 
It will not be evaluated by eliciting persons' preferences in the 
matter • 
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